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ABSTRACT 



A search is conducted for a target moving in discrete time among a finite number of cells according to 
a known Markov process. The searcher must choose one cell in which to search in each time period. The set 
of cells available for search depends upon the cell chosen in the last time period. The problem is to find a 
search path, i.e., a sequence of search cells, that maximizes the probability of detecting the target in a fixed 
number of time periods. Closely following earher work by Theodor Stewart, a branch-and-bound procedure 
is developed which finds optimal search paths. This procedure is tested and appears to be more efficient 
than existing dynamic programming solution methods. 
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Constrained Path, Moving Target Search Problem 

by 

James N. Eagle 

Department of Operations Research 
Naval Postgraduate School 
Monterey, CA 93943 

James R. Yee 

Department of Electrical Engineering-Systems 
University of Southern Cahfornia 
Los Angeles, CA 90089 



A searcher and target move among a finite set of cells C = 1,2, .,,, N in discrete time. At the beginning 
of each time period, one cell is searched. If the target is in the selected cell i, it is detected with probability 
qi. If the target is not in the cell searched, it can not be detected during the current time period. After 
each search, a target in cell j moves to cell k with probability pj^. The target transition matrix, P = [pjk] 
is known to the searcher. The searcher’s path is constrained in that if the searcher is currently in cell f, the 
next search cell must be selected from a set of neighboring cells C{. (The set Ci is also those cells from which 
i can be reached.) The object of the search is to minimize the probability of not detecting the target in T 
searches. 



1. Background 

The path constrained search problem, described above, is a difficult one to solve efficiently. Trummel and 
Weisinger [1986] showed that the path constrained search problem with a stationary target is NP-complete. 
The moving target problem, which is a generalization of the stationary target problem, is easily shown to 
be at least as difficult. 

The only optimal solution technique mentioned in the literature for this problem has been the dynamic 
programming procedure of Eagle [1984a]. Although this method can solve problems much more quickly than 
can total enumeration, it can require a large amount of computer storage as problem size increases. 

There have been several approximate solution procedures suggested for this problem. The first such 
method proposed was a branch-and-bound method by Stewart [1979] and [1980]. Stewart’s bounds were 
obtained by solving an integer problem without path constraints. The solution to this simpler problem was 
obtained using a discrete version of the moving target search algorithm given by Brown [1980]. However, 
Brown’s algorithm does not necessarily give optimal solutions when search effort is discrete, so these “bounds” 
are only approximate and can result in an optimal branch of the enumeration tree being mistakenly fathomed. 
Nonetheless, Stewart’s computational experience with 1-dimensional search problems indicates that the 
method can perform well. 

Another approximate procedure was given by Eagle [1984b]. This dynamic programming method uses 
a moving or “rolling” time horizon that greatly reduces computer storage requirements. It was used to 
approximately solve a small 2-dimensional problem (3 by 3 search grid) for 40 time periods. This procedure 
generalizes myopic search by selecting the next cell to be searched under the assumption that the search 
ends m time periods in the future. For myopic search, m is 1. For small enough m, this procedure can be 
implemented on a microcomputer. 
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Reported here is an optimal solution method which appears to perform more efficiently than the existing 
optimal dynamic programming procedures. This work is a direct continuation of that of Stewart [1979]. The 
approximate bound in Stewart’s branch-and-bound procedure is replaced with a true lower bound, thus 
guaranteeing optimal solutions. The true bound is obtained by relaxing the integer problem in a manner 
first suggested by Stewart [1979]. The branch-and-bound procedure is made computationally feasible by 
special structure in the relaxed problem which allows for its efficient solution. 

2. The Relaxed Problem 

The procedure presented here depends critically on the efficient solution of a convex nonhnear program, 
here called the relaxed problem or RP. The relaxed problem, introduced by Stewart [1979], allows infinite 
divisibility of search effort over a subset of the search cells, while choosing this subset properly to maintain 
a representation of path constraints. Following Stewart, let x(z, j, <) be the amount of search effort that is 
redistributed from cell i in time period i to cell j in time period < + 1. And let X{i^t) be the total search 
effort in cell i at time t. Then, 

^ x(Ar,f,f — 1), < = 1,2,...,T and 2 = 1 ,...,//. 

keCi 

The searcher’s initial search effort distribution is X{i, 0), and is assumed to be specified. For a single searcher 
(our assumption here), we additionally require A“(i,0) = 1 if i is the searcher’s starting cell, and AT(i,0) = 0 
otherwise. 

An exponential detection function is assumed for RP. That is, if the search effort in cell i at time t is 
AT(i,<), then the probability of detecting the target during that search is 1 — exp(— a,*Ar(i, <)). The term a,- 
is selected so that 1 — exp(— Oj) = g,-. This insures that integer solutions for RP have the correct objective 
function values. To keep cti finite, we require qi < 1. Additionally we let u; = (o;(l),u;(2), . . . ,a;(T)) be 
a sample target track, and be the probability that the target follows that path. Finally, the set of all 
possible target paths is Q. 

The optimal search flows for RP are obtained by solving the following nonlinear program: 

T 

minimize ^ p(w) exp|- ^ a„(t) ^ x(i, w(<), t - 1)} 

wen t=i 

subject to: 

j€Ci 

^ x{j,k,t) = 0, j = l,...,N and < = 1,...,T-1 

ieCj keCj 

2 ,j = l,...,iV and <=0, ...,T— 1 

As Stewart observed, the objective function for RP is convex, and the constraints are those of a network. 
The integer problem (IP) is identical to RP except that each x(i, j, <) must be either 1 or 0. 

When the target motion is Markovian with transition matrix P = [p»j], the objective function of RP 
can be written 

minimize 7 t(1)PiP2 • • • (1) 

where 7t(1) is a row vector of the target’s initial distribution over the search cells C (assumed to be known), 
1 is a column vector of ones, and Pt is the target transition matrix P with each row i multiplied by 
exp(— QfjAr(i, <)). The matrix Pt is substochastic and updates the (defective) target distribution for the 
search in time period t and the target transition into time period t + 1, 
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As Brown [1980] pointed out, the objective function (1) can be written by conditioning on the target’s 
cell in time period t. Specifically, 

N 

^r(j,<)exp|- ^ (2) 

;=i »€Cj- 

where r{jyt) is the probability of the target reaching cell j in time period t without being detected by searches 
in time periods 1,2,...,^ — 1; and s{j,t) is the probability of a target which starts in cell j in time period t 
surviving the searches in time periods < + 1, Both and 5(-,<) are easily ccJculated from r(*,i — 1) 

and s{-,t + 1), respectively. The recursions begin with r(i, 1) = ^»(1) and s(i,T) = 1, for all cells i. 

The importance of expressing the objective function as in (2) is that it allows a convenient calculation 
of the gradient. If /(x) is the objective function, then elements of the gradient V/(x) are 

■^p^$- = -ajr{j,t + l)exp(- ^ j, < + 1). 

Note that all gradient elements are nonpositive, since increasing the search flow along any arc can not increase 
the probability of missing the target. 



3. Solving the Relaxed Problem 

Stewart solves RP essentially as suggested by Frank and Wolfe [1956]. That is, starting with a set 
of feasible search flows xi, the gradient V/(xi) is computed. Then x[ is found by minimizing the linear 
constraint V/(xi)x subject to the network constraints of RP. An improved, feasible set of flows Xo is then 
found by searching along the line segment [xj, x\]. The process continues, generating an improving sequence 
of flows Xi , X2, X3 , . . . until a stopping criterion is met. 

Stewart’s solution procedure for RP is particularly effective for the search problem presented here for 
two reasons: 

1. Since this is a single-searcher problem, arc capacity constraints (allowed by Stewart) are not neces- 
sary. Also a single initial search cell is specified, rather than a search effort distribution over several cells. 
Consequently, the linear subproblem reduces to an acyclic, shortest path problem — perhaps the simplest of 
all nontrivial hnear programs to solve. The solution can be obtained by dynamic programming in polyno- 
mial time (each arc is examined only once) and does not require a general network solver. We note that the 
network is acyclic because each arc connects a time i node to a time < -{- 1 node. 

2. At each iteration, the Frank- Wolfe method produces a lower bound for the optimal RP objective 
function value. In particular, if x^i is the current feasible solution to RP, and x"* is the optimal solution, 
then by the convexity of /(x), 

/(x*) > /(x„) + V/(x„)(x* - x„)‘. 

And since x^ minimizes V/(xi)x over all feasible flows x, 

/(x*) > /(x„) + V/(x„)(x;, - x„)‘. 

For convex programs with bounded feasible regions (as we have here), Frank and Wolfe [1956] showed that 
this lower bound for /(x*) becomes exact as x„ approaches x*. The bound is important since a lower bound 
to IP (which is provided by a lower bound to RP) is required for the branch-and-bound solution of IP. 
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4. Solving the Integer Problem 



Stewart [1979] and [1980] gave a branch-and-bound algorithm for IP. The only change suggested here is 
the use of RP for fathoming. There are, however, at least two important implementation issues. First, when 
using Stewart^s procedure to solve a particular instance of RP, the Frank-Wolfe steps need continue only until 
branching or fathoming criteria are met. In particular, let PBEST be the branch-and-bound procedure’s 
current best (i.e., smallest) objective function value for an integer solution. Then during the solution of RP, 

1. If the lower bound to RP (/(x„) -h V/(x„)(x^ — x„)^) becomes greater than PBEST, the current 
branch is fathomed. 

2. If the upper bound to RP (/(x^)) becomes smaller than PBEST, then fathoming is impossible, and 
the procedure must branch one time period deeper into the enumeration tree. 

3. If the difference between the current RP upper and lower bounds (V/(xn)(xJ^ — x^)*) becomes less 
than some specified e > 0, then the procedure branches. 

A second observation is that each xj^, which is an optimal solution to a linear subproblem, is an extreme 
point of the feasible region for RP. Each x^ is thus integer-valued and a feasible solution to IP, These 
solutions can be used to obtain an initial, integer starting solution and to continually update PBEST in the 
branch-and-bound procedure. 



6. Computational Results 

In Eagle [1984a], a 9-cell, 10-time period problem was reported solved to optimality using dynamic 
programming, and requiring 19.3 CPU minutes on a mainframe computer. This same problem required 2.5 
CPU minutes using the branch-and-bound procedure presented here. (The comparison is not entirely fair 
since the dynamic programming solution was, essentially, for all possible initial target distributions. Once an 
optimal solution was obtained for any initial target distribution, only an additional 1.5 CPU seconds were 
required to obtain an optimal solution for any other initial target distribution.) A few larger problems have 
been attempted. A 25-cell, 10-time period problem required 3.15 CPU minutes; and a 49-cell, 10-time period 
problem required 4.7 CPU minutes. All computations were done in Fortran 77 on the Naval Postgraduate 
School IBM 3033 mainframe computer. 
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